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ABSTRACT 

This study tried to determine how well a combination 
of junior/senior undergraduate grade point average, standardized test 
scores, and writing sample scores can predict a student's success in 
a graduate College of Education. The study used data on 164 students 
selected to have a slight over-representation of students with lower 
Miller Analogies Test (MAT) scores. University administrative offices 
were able to supply a writing sample for 151 of the 164 students 
limiting the sample to those 151 students. Writing samples were 
scored by 3 readers with several years experience scoring essays. 
Predictor variables were undergraduate grade point average from the 
junior and senior years, the MAT score, and the writing sample score. 
Dependent variables were student teaching grade, graduate grade point 
average, and graduation success. Data analysis indicated that 
although none of the variables could accurately predict student 
teaching performance or graduation success, the writing sample, in 
conjunction with the junior/senior undergraduate grade point average 
could be used with moderate success to predict a student's graduate 
grade point average. Contains six references. (JB) 
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Objective 



The objective of this study was to determine how well a combination of 
junior/senior undergraduate grade point average, Miller Analogies Test (MAT) 
scores, andwriting sample scores can predict a student's success in a graduate 
College of Education. For this study, success is defined as (1) the student's 
grade in student teaching, (2) the student's graduate GPA and (3) whether or not 
the student graduated from the program. 

Perspectives 

Research studies have indicated that standardized test scores are of 
limited use to an admissions office wanting to predict performance of applicants 
to graduate Schools of Education, For the past thirty years the usefulness of 
standardized tests, such as the MAT, as predictors of ability in graduate school 
has been limited (Ainsworth and Fox, 1966; Gill and Marascuilo, 1967), Payne and 
Tuttle (1966) found predictive validity coefficients for the Miller Analogies 
Test that ranged from ,00 to ,59, A more recent study (de-Felix and Houston, 
1986) indicates that there is a weak association between standardized entrance 
tests and indicators of success, particularly for minorities. Today, with the 
movement toward assessment devices which are more authentic and naturalistic in 
nature, it seems reasonable to explore the possibility of using a writing sample 
as a replacement for a standardized test that is used as an admission criterion. 

Currently, the MAT score and a student's junior/senior undergraduate GPA 
are used as two of the criteria for admission to graduate study in the College 
of Education at a metropolitan university in the Midwest, The Office of 
Institutional Research at the university recently conducted a study which 
examined the relationship between graduate GPA and both junior/senior 
undergraduate GPA and MAT scores (Office of Institutional Research, 1992), The 
study found that junior/senior undergraduate GPA and the MAT scores have only a 
limited amount of relationship (r = ,29 and r = .30, respectively) to graduate 
grade point average. This current study extends the previous study by (1) 
including a writing sample as a predictor of graduate grade point average and (2) 
considering a student's grade in student teaching and whether or not the student 
graduated from the program as additional dependent variables. 

Methods 

The Office of Institutional Research at the university supplied a list of 
nearly 1,500 Master of Arts in Teaching students who were taking courses in the 
College of Education from September 1989 to August 1991, The list of students 
was then stratified with respect to MAT scores, A systematic sample of 164 
students was selected with a slight over-representation of students with MAT 
scores of 29 and below to assure enough subjects were at the lower end of the MAT 
scale. 
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'A writing sample for each of the identified students was then collected 
from the administrative offices at the university. Writing samples could not be 
located for 13 (8%) students. Thus, the final sample consisted of 151 students. 
Data for the remaining variables were collected from the university's Office of 
Institutional Research. 

D ata Sources 

Predictor Variables 

Junior/Senior Undergraduate 6PA . This first predictor was determined by 
averaging the grade point values (A 4, B = 3, C = 2, D = 1, F = 0) for all 
courses taken during the student's junior and senior years in undergraduate 
school . 

Miller Analogies Test (MAT) Score . The raw score from the Miller Analogies 
T est was used as a second predictor in this study. Raw scores on the MAT may 
range from 0 to 100. 

Writing Sample Score . As part of the admission process for graduate study, 
applicants complete a written statement, which in the past was read, but received 
less weight than other factors in determining admission to the graduate school. 
The following statement appears on the application form: 

Please describe your reasons for wishing to complete the 
degree for which you are applying. Present some of your 
future professional plans and indicate what you h~?pe to 
accomplish from this additional study. This statement is 
one of the criteria used to evaluate your application. 

The writing samples were scored independently by 3 readers who have had 
several years experience scoring essays which are used as an admission criterion 
for alternative teacher certification programs in Texas. The writing samples 
were scored according to the following 4-point scale which is a modification cf 
the scale used to evaluate essays written by applicants to the Texas alternative 
teacher certification programs: 

4 - A well organized essay that demonstrates clear competence 
in writing, although it may have a few minor errors or 
occasional awkwardness of expression. 

3 - An essay that is well-written but somewhat general and 
lacking in details. It may have 1 or 2 major errors and 
some minor errors. 

2 - The essay may be too brief, underdeveloped or too general 
to deal adequately with the topic, or the essay may indi- 
cate problems in basic writing skills. 

1 - The essay displays serious faults in writing and weakness 
in content. 

To score the writing samples each reader assigned one of the four score 
points to the essay and was also allowed to indicate a plus +) or a minus (-) 
if the scorer thought that the writing sample was at the top or at the bottom of 
a particular score category. In only 2 instances out of 151 did the scores of 
any two readers differ by more than 1 point. In those two cases the difference 
was resolved so that all readp- s arrived at scores that were no more than 1 score 
point apart. 



ERIC 



3 



To increase the variability in the essay score scale, the scores of 'the 3 
readers were summed and multiplied by 4, and then one point was added to the 
result for each plus ( + ) given by the readers (maximum of 3 points) , and one 
point was subtracted from the result for each minus (-) given by the readers 
(maximum of 3 points). Thus, the possible range of scores for the writing sample 
was 12 to 51. The interrater reliability was r = .92 for this study. 

Dependent Variables 

Grade in Student Teaching . To receive the Master of Arts in Teaching 
degree, each student must complete a term of student teaching. For analysis 

purposes, the student teaching grade was measured using the following scale: (A 
= 4, B = 3, C = 2, D = 1, F = 0). 

Graduate GPA . For each student in the sample the grade point average was 
determined by averaging the grade point value using the scale above for all 
graduate courses taken in the college. 

Graduation Success . If a student had graduated by June 1993, the student 
was assigned a code of 1 for the graduate success variable. If a student had not 
graduated by June 1993, the student was assigned a 0 for the graduation success 
variable. 

Results 

Means and standard deviations were determined for the MAT and the GPA 
variables. In addition, correlations among all the predictor and dependent 
variables were calculated. The mean MAT score reflected the slight over-sampling 
of students with lower MAT scores (sample mean = 45, population mean =47). A 
similar finding was also evident for graduate GPA (sample mean = 3.54, population 
mean = 3.86) and junior/senior GPA (sample mean = 2.87, population mean = 3.09). 

The only significant correlations (g < .01) between the predictors and the 
dependent variables were between graduate GPA and the following predictor 
variables: MAT scores (r = .26), junior/senior GPA (r = .37) and writing sample 
scores (r = .40) (see Table 1). Based on this finding, a stepwise regression was 
performed using graduate GPA as the dependent variable and MAT scores, 
junior/senior GPA and the wr .ting sample scores as predictors* The results of 
the regression analysis indicated that a combination of writing sample scores and 
junior/senior GPA resulted in multiple R of .48* MAT scores did not 
significantly contribute to the regression equatidn. 

Further investigation of this relationship between graduate GPA and (1) 
writing sample scores, (2) junior/senior undergraduate GPA, and (3) MAT scores- 
indicated that students with an average graduate GPA of 3.00 or above had a mean 
essay score of 2.35 (SD = .64) on the 4-point scale, a mean junior/senior 
undergraduate GPA of 2.90 (SD = .45), and an average MAT score of 45.6 (SD = 
18.0) (see Table 2). Those students with an average graduate GPA of below 3.00, 
which is required for graduation, had a moan essay score of 1.76 (SD = .42) on 
the 4-point scale, a mean junior/senior undergraduate GPA of 2.50 (SD = .30), and 
an average MAT score of 34.5 (SD = 17.8). These differences between the two 
groups were significant for the essay scores (t = 3.49 £ < .001), the 
junior/senior undergraduate GPA (t = 2.88, a < -01)and the MAT (t = 2.26, £ < 
.05). Thus, the writing sample and junior/senior GPA were found to be the most 
effective variables for distinguishing between the students who were eligible and 
the students who were not eligible for graduation. 
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Finally, to investigate further the relationship between the predictor variables 
an' nrogram completion, the students were divided into four groups: (1) those 
Wi had graduated by August 1993 (Graduated); (2) those continuing to take 
courses between August 1992 and August 1993 (Continuing); (3) those who had taken 
no courses between August 1992 and August 1993 (Stopped-out> ; (4) those who had 
been dropped from the program because of academic reasons (Dropped). The results 
of these comparisons, shown in Table 3, indicated that while all three dependent 
variables could significantly differentiate the group of students who had 
graduated from the group of students who had been dropped from the program for 
academic reasons, only the writing sample was able to significantly differentiate 
the group of students who were continuing to take courses from the group of 
students who had been dropped from the program for academic reasons. 

Educatio,... 1 Importance 

The data from this study indicate that although none of the variables could 
accurately predict student teaching performance or graduation success, the 
writing sample, in conjunction with the junior/senior undergraduate GPA, could 
be used with moderate success to predict a student's graduate GPA. If the 
results of this study could be replicated with students seeking a Masters of Arts 
in Teaching degree and with students seeking other masters degrees in Education, 
then Colleges of Education may choose to include the evaluation of writing 
samples as a more authentic and valid method of assessing the ability of students 
to dG graduate work. 
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Table 1 



Correlations Between the Predictor and Criterion Variables 



Predictors 



Criterion Variables 



Student 
Teaching 
Grade 



Program 
Completion 
(Graduation) 



Graduate 
Education 
GPA 



Writing Sample 
JrSr GPA 
Millers Test 



,17 
.15 
,09 



.05 
.13 
.09 



.40** 
.37** 
.25** 



**p < .01; n = 151 

Multiple R for Writing Sample and JrSr GPA predicting Graduate Education GPA is 
R = .48. 



Table 2 

Performance on the Predictor Variables 
For Students with Graduate GPA' s Above and Below 3.00 

Students with Students with 

Graduate GPAs > 3.00 Graduate GPAs < 3.00 



Predictors 

mean SD n mean SD n t 

Writing Sample 2.35 .54 135 1.76 .42 15 3.49*** 

JrSr GPA 2.90 .45 131 2.50 .30 11 2.88** 

Millers Test 45.6 18.0 133 34.. 17.8 15 2.26* 

*** p < .001, ** p < .01, * p < .05 
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Table 3 



Predictors 



Performance on the Predictor Variables for 
Four Categories of Student Status 



Student Status 



Graduated Continuing Stopped-out Dropped 



Writing Sample mean 2.38* 2.33* 2.25 1.71* 

SD .67 .67 .58 .42 

n 71 24 46 10 

JrSr GPA mean 2.96* 2.94 2.78 2.49* 

SD .45 .41 .46 .30 

n 65 23 45 9 

Millers Test mean 47.5* 38.9 45.5 31.6* 

SD 19.7 15.0 17.3 9.9 

n 69 23 46 10 

*p < .05; significant differences are shown for each row. 

Note: For the Writing Sample variable, the Graduated and Continuing students did 

not score significantly different from each other but did score significantly 
higher than the Dropped students. 
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